Hybrid cloud and cluster computing paradigms for life science applications
Identifieur interne : 000892 ( Main/Exploration ); précédent : 000891; suivant : 000893Hybrid cloud and cluster computing paradigms for life science applications
Auteurs : Judy Qiu [États-Unis] ; Jaliya Ekanayake [États-Unis] ; Thilina Gunarathne [États-Unis] ; Jong Youl Choi [États-Unis] ; Seung-Hee Bae [États-Unis] ; Hui Li [États-Unis] ; Bingjing Zhang [États-Unis] ; Tak-Lon Wu [États-Unis] ; Yang Ruan [États-Unis] ; Saliya Ekanayake [États-Unis] ; Adam Hughes [États-Unis] ; Geoffrey Fox [États-Unis]Source :
- BMC Bioinformatics [ 1471-2105 ] ; 2010.
Abstract
Clouds and MapReduce have shown themselves to be a broadly useful approach to scientific computing especially for parallel data intensive applications. However they have limited applicability to some areas such as data mining because MapReduce has poor performance on problems with an iterative structure present in the linear algebra that underlies much data analysis. Such problems can be run efficiently on clusters using MPI leading to a hybrid cloud and cluster environment. This motivates the design and implementation of an open source Iterative MapReduce system Twister.
Comparisons of Amazon, Azure, and traditional Linux and Windows environments on common applications have shown encouraging performance and usability comparisons in several important non iterative cases. These are linked to MPI applications for final stages of the data analysis. Further we have released the open source Twister Iterative MapReduce and benchmarked it against basic MapReduce (Hadoop) and MPI in information retrieval and life sciences applications.
The hybrid cloud (MapReduce) and cluster (MPI) approach offers an attractive production environment while Twister promises a uniform programming environment for many Life Sciences applications.
We used commercial clouds Amazon and Azure and the NSF resource FutureGrid to perform detailed comparisons and evaluations of different approaches to data intensive computing. Several applications were developed in MPI, MapReduce and Twister in these different environments.
Url:
DOI: 10.1186/1471-2105-11-S12-S3
PubMed: 21210982
PubMed Central: 3040529
Affiliations:
Links toward previous steps (curation, corpus...)
- to stream Pmc, to step Corpus: 000484
- to stream Pmc, to step Curation: 000484
- to stream Pmc, to step Checkpoint: 000577
- to stream Ncbi, to step Merge: 000188
- to stream Ncbi, to step Curation: 000188
- to stream Ncbi, to step Checkpoint: 000188
- to stream Main, to step Merge: 000895
- to stream Main, to step Curation: 000892
Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">Hybrid cloud and cluster computing paradigms for life science applications</title><author><name sortKey="Qiu, Judy" sort="Qiu, Judy" uniqKey="Qiu J" first="Judy" last="Qiu">Judy Qiu</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Ekanayake, Jaliya" sort="Ekanayake, Jaliya" uniqKey="Ekanayake J" first="Jaliya" last="Ekanayake">Jaliya Ekanayake</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Gunarathne, Thilina" sort="Gunarathne, Thilina" uniqKey="Gunarathne T" first="Thilina" last="Gunarathne">Thilina Gunarathne</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Choi, Jong Youl" sort="Choi, Jong Youl" uniqKey="Choi J" first="Jong Youl" last="Choi">Jong Youl Choi</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Bae, Seung Hee" sort="Bae, Seung Hee" uniqKey="Bae S" first="Seung-Hee" last="Bae">Seung-Hee Bae</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Li, Hui" sort="Li, Hui" uniqKey="Li H" first="Hui" last="Li">Hui Li</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Zhang, Bingjing" sort="Zhang, Bingjing" uniqKey="Zhang B" first="Bingjing" last="Zhang">Bingjing Zhang</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Wu, Tak Lon" sort="Wu, Tak Lon" uniqKey="Wu T" first="Tak-Lon" last="Wu">Tak-Lon Wu</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Ruan, Yang" sort="Ruan, Yang" uniqKey="Ruan Y" first="Yang" last="Ruan">Yang Ruan</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Ekanayake, Saliya" sort="Ekanayake, Saliya" uniqKey="Ekanayake S" first="Saliya" last="Ekanayake">Saliya Ekanayake</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Hughes, Adam" sort="Hughes, Adam" uniqKey="Hughes A" first="Adam" last="Hughes">Adam Hughes</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Fox, Geoffrey" sort="Fox, Geoffrey" uniqKey="Fox G" first="Geoffrey" last="Fox">Geoffrey Fox</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author></titleStmt><publicationStmt><idno type="wicri:source">PMC</idno><idno type="pmid">21210982</idno><idno type="pmc">3040529</idno><idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3040529</idno><idno type="RBID">PMC:3040529</idno><idno type="doi">10.1186/1471-2105-11-S12-S3</idno><date when="2010">2010</date><idno type="wicri:Area/Pmc/Corpus">000484</idno><idno type="wicri:Area/Pmc/Curation">000484</idno><idno type="wicri:Area/Pmc/Checkpoint">000577</idno><idno type="wicri:Area/Ncbi/Merge">000188</idno><idno type="wicri:Area/Ncbi/Curation">000188</idno><idno type="wicri:Area/Ncbi/Checkpoint">000188</idno><idno type="wicri:Area/Main/Merge">000895</idno><idno type="wicri:Area/Main/Curation">000892</idno><idno type="wicri:Area/Main/Exploration">000892</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">Hybrid cloud and cluster computing paradigms for life science applications</title><author><name sortKey="Qiu, Judy" sort="Qiu, Judy" uniqKey="Qiu J" first="Judy" last="Qiu">Judy Qiu</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Ekanayake, Jaliya" sort="Ekanayake, Jaliya" uniqKey="Ekanayake J" first="Jaliya" last="Ekanayake">Jaliya Ekanayake</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Gunarathne, Thilina" sort="Gunarathne, Thilina" uniqKey="Gunarathne T" first="Thilina" last="Gunarathne">Thilina Gunarathne</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Choi, Jong Youl" sort="Choi, Jong Youl" uniqKey="Choi J" first="Jong Youl" last="Choi">Jong Youl Choi</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Bae, Seung Hee" sort="Bae, Seung Hee" uniqKey="Bae S" first="Seung-Hee" last="Bae">Seung-Hee Bae</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Li, Hui" sort="Li, Hui" uniqKey="Li H" first="Hui" last="Li">Hui Li</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Zhang, Bingjing" sort="Zhang, Bingjing" uniqKey="Zhang B" first="Bingjing" last="Zhang">Bingjing Zhang</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Wu, Tak Lon" sort="Wu, Tak Lon" uniqKey="Wu T" first="Tak-Lon" last="Wu">Tak-Lon Wu</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Ruan, Yang" sort="Ruan, Yang" uniqKey="Ruan Y" first="Yang" last="Ruan">Yang Ruan</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Ekanayake, Saliya" sort="Ekanayake, Saliya" uniqKey="Ekanayake S" first="Saliya" last="Ekanayake">Saliya Ekanayake</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Hughes, Adam" sort="Hughes, Adam" uniqKey="Hughes A" first="Adam" last="Hughes">Adam Hughes</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author><author><name sortKey="Fox, Geoffrey" sort="Fox, Geoffrey" uniqKey="Fox G" first="Geoffrey" last="Fox">Geoffrey Fox</name><affiliation wicri:level="2"><nlm:aff id="I1">School of Informatics and Computing, Indiana University, Bloomington, IN 47405, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>School of Informatics and Computing, Indiana University, Bloomington, IN 47405</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation><affiliation wicri:level="2"><nlm:aff id="I2">Pervasive Technology Institute, Indiana University, Bloomington, IN 47408, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Pervasive Technology Institute, Indiana University, Bloomington, IN 47408</wicri:regionArea><placeName><region type="state">Indiana</region></placeName></affiliation></author></analytic><series><title level="j">BMC Bioinformatics</title><idno type="eISSN">1471-2105</idno><imprint><date when="2010">2010</date></imprint></series></biblStruct></sourceDesc></fileDesc><profileDesc><textClass></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en"><sec><title>Background</title><p>Clouds and MapReduce have shown themselves to be a broadly useful approach to scientific computing especially for parallel data intensive applications. However they have limited applicability to some areas such as data mining because MapReduce has poor performance on problems with an iterative structure present in the linear algebra that underlies much data analysis. Such problems can be run efficiently on clusters using MPI leading to a hybrid cloud and cluster environment. This motivates the design and implementation of an open source Iterative MapReduce system Twister.</p></sec><sec><title>Results</title><p>Comparisons of Amazon, Azure, and traditional Linux and Windows environments on common applications have shown encouraging performance and usability comparisons in several important non iterative cases. These are linked to MPI applications for final stages of the data analysis. Further we have released the open source Twister Iterative MapReduce and benchmarked it against basic MapReduce (Hadoop) and MPI in information retrieval and life sciences applications.</p></sec><sec><title>Conclusions</title><p>The hybrid cloud (MapReduce) and cluster (MPI) approach offers an attractive production environment while Twister promises a uniform programming environment for many Life Sciences applications.</p></sec><sec><title>Methods</title><p>We used commercial clouds Amazon and Azure and the NSF resource FutureGrid to perform detailed comparisons and evaluations of different approaches to data intensive computing. Several applications were developed in MPI, MapReduce and Twister in these different environments.</p></sec></div></front><back><div1 type="bibliography"><listBibl><biblStruct><analytic><author><name sortKey="Armbrust, M" uniqKey="Armbrust M">M Armbrust</name></author><author><name sortKey="Fox" uniqKey="Fox">Fox</name></author><author><name sortKey="Griffith, R" uniqKey="Griffith R">R Griffith</name></author><author><name sortKey="Joseph, Ad" uniqKey="Joseph A">AD Joseph</name></author><author><name sortKey="Katz, R" uniqKey="Katz R">R Katz</name></author><author><name sortKey="Konwinski, A" uniqKey="Konwinski A">A Konwinski</name></author><author><name sortKey="Lee, G" uniqKey="Lee G">G Lee</name></author><author><name sortKey="Patterson, D" uniqKey="Patterson D">D Patterson</name></author><author><name sortKey="Rabkin, A" uniqKey="Rabkin A">A Rabkin</name></author><author><name sortKey="Stoica, I" uniqKey="Stoica I">I Stoica</name></author><author><name sortKey="Zaharia, M" uniqKey="Zaharia M">M Zaharia</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Walker, E" uniqKey="Walker E">E Walker</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ekanayake, J" uniqKey="Ekanayake J">J Ekanayake</name></author><author><name sortKey="Qiu, Xh" uniqKey="Qiu X">XH Qiu</name></author><author><name sortKey="Gunarathne, T" uniqKey="Gunarathne T">T Gunarathne</name></author><author><name sortKey="Beason, S" uniqKey="Beason S">S Beason</name></author><author><name sortKey="Fox, G" uniqKey="Fox G">G Fox</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Evangelinos, C" uniqKey="Evangelinos C">C Evangelinos</name></author><author><name sortKey="Hill, Cn" uniqKey="Hill C">CN Hill</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Lange, J" uniqKey="Lange J">J Lange</name></author><author><name sortKey="Pedretti, K" uniqKey="Pedretti K">K Pedretti</name></author><author><name sortKey="Hudson, T" uniqKey="Hudson T">T Hudson</name></author><author><name sortKey="Dinda, P" uniqKey="Dinda P">P Dinda</name></author><author><name sortKey="Cui, Z" uniqKey="Cui Z">Z Cui</name></author><author><name sortKey="Xia, L" uniqKey="Xia L">L Xia</name></author><author><name sortKey="Bridges, P" uniqKey="Bridges P">P Bridges</name></author><author><name sortKey="Gocke, A" uniqKey="Gocke A">A Gocke</name></author><author><name sortKey="Laconette, S" uniqKey="Laconette S">S laconette</name></author><author><name sortKey="Levenhagen, M" uniqKey="Levenhagen M">M Levenhagen</name></author><author><name sortKey="Brightwell, R" uniqKey="Brightwell R">R Brightwell</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Fox, G" uniqKey="Fox G">G Fox</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Dean, J" uniqKey="Dean J">J Dean</name></author><author><name sortKey="Ghemawat, S" uniqKey="Ghemawat S">S Ghemawat</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Ekanayake, J" uniqKey="Ekanayake J">J Ekanayake</name></author><author><name sortKey="Gunarathne, T" uniqKey="Gunarathne T">T Gunarathne</name></author><author><name sortKey="Qiu, J" uniqKey="Qiu J">J Qiu</name></author><author><name sortKey="Fox, G" uniqKey="Fox G">G Fox</name></author><author><name sortKey="Beason, S" uniqKey="Beason S">S Beason</name></author><author><name sortKey="Choi, Jy" uniqKey="Choi J">JY Choi</name></author><author><name sortKey="Ruan, Y" uniqKey="Ruan Y">Y Ruan</name></author><author><name sortKey="Bae, Sh" uniqKey="Bae S">SH Bae</name></author><author><name sortKey="Li, H" uniqKey="Li H">H Li</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ekanayake, J" uniqKey="Ekanayake J">J Ekanayake</name></author><author><name sortKey="Balkir, A" uniqKey="Balkir A">A Balkir</name></author><author><name sortKey="Gunarathne, T" uniqKey="Gunarathne T">T Gunarathne</name></author><author><name sortKey="Fox, G" uniqKey="Fox G">G Fox</name></author><author><name sortKey="Poulain, C" uniqKey="Poulain C">C Poulain</name></author><author><name sortKey="Araujo, N" uniqKey="Araujo N">N Araujo</name></author><author><name sortKey="Barga, R" uniqKey="Barga R">R Barga</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Fox, Gc" uniqKey="Fox G">GC Fox</name></author><author><name sortKey="Qiu, Xh" uniqKey="Qiu X">XH Qiu</name></author><author><name sortKey="Beason, S" uniqKey="Beason S">S Beason</name></author><author><name sortKey="Choi, Jy" uniqKey="Choi J">JY Choi</name></author><author><name sortKey="Rho, M" uniqKey="Rho M">M Rho</name></author><author><name sortKey="Tang, H" uniqKey="Tang H">H Tang</name></author><author><name sortKey="Devadasan, N" uniqKey="Devadasan N">N Devadasan</name></author><author><name sortKey="Liu, G" uniqKey="Liu G">G Liu</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Bae, Sh" uniqKey="Bae S">SH Bae</name></author><author><name sortKey="Choi, Jh" uniqKey="Choi J">JH Choi</name></author><author><name sortKey="Qiu, J" uniqKey="Qiu J">J Qiu</name></author><author><name sortKey="Fox, J" uniqKey="Fox J">J Fox</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Sammon, Jw" uniqKey="Sammon J">JW Sammon</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ekanayake, J" uniqKey="Ekanayake J">J Ekanayake</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Fox, Gc" uniqKey="Fox G">GC Fox</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Fox, Gc" uniqKey="Fox G">GC Fox</name></author><author><name sortKey="Williams, Rd" uniqKey="Williams R">RD Williams</name></author><author><name sortKey="Messina, Pc" uniqKey="Messina P">PC Messina</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Chu" uniqKey="Chu">Chu</name></author><author><name sortKey="Cheng, T" uniqKey="Cheng T">T Cheng</name></author><author><name sortKey="Sang, Kim K" uniqKey="Sang K">Kim K Sang</name></author><author><name sortKey="Yi, Lin A" uniqKey="Yi L">Lin A Yi</name></author><author><name sortKey="Yu, Y" uniqKey="Yu Y">Y Yu</name></author><author><name sortKey="Bradski, R" uniqKey="Bradski R">R Bradski</name></author><author><name sortKey="Ng, A" uniqKey="Ng A">A Ng</name></author><author><name sortKey="Olukotun, K" uniqKey="Olukotun K">K Olukotun</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Qiu, X" uniqKey="Qiu X">X Qiu</name></author><author><name sortKey="Ekanayake, J" uniqKey="Ekanayake J">J Ekanayake</name></author><author><name sortKey="Beason, S" uniqKey="Beason S">S Beason</name></author><author><name sortKey="Gunarathne, T" uniqKey="Gunarathne T">T Gunarathne</name></author><author><name sortKey="Fox, G" uniqKey="Fox G">G Fox</name></author><author><name sortKey="Barga, R" uniqKey="Barga R">R Barga</name></author><author><name sortKey="Gannon, D" uniqKey="Gannon D">D Gannon</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Ekanayake, J" uniqKey="Ekanayake J">J Ekanayake</name></author><author><name sortKey="Li, Hui" uniqKey="Li H">Hui Li</name></author><author><name sortKey="Bingjing, B Zhang" uniqKey="Bingjing B">B Zhang Bingjing</name></author><author><name sortKey="Gunarathne, T" uniqKey="Gunarathne T">T Gunarathne</name></author><author><name sortKey="Bae, Sh" uniqKey="Bae S">SH Bae</name></author><author><name sortKey="Qiu, J" uniqKey="Qiu J">J Qiu</name></author><author><name sortKey="Fox, G" uniqKey="Fox G">G Fox</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Gunarathne, T" uniqKey="Gunarathne T">T Gunarathne</name></author><author><name sortKey="Wu, Tl" uniqKey="Wu T">TL Wu</name></author><author><name sortKey="Qiu, J" uniqKey="Qiu J">J Qiu</name></author><author><name sortKey="Fox, G" uniqKey="Fox G">G Fox</name></author></analytic></biblStruct></listBibl></div1></back></TEI><affiliations><list><country><li>États-Unis</li></country><region><li>Indiana</li></region></list><tree><country name="États-Unis"><region name="Indiana"><name sortKey="Qiu, Judy" sort="Qiu, Judy" uniqKey="Qiu J" first="Judy" last="Qiu">Judy Qiu</name></region><name sortKey="Bae, Seung Hee" sort="Bae, Seung Hee" uniqKey="Bae S" first="Seung-Hee" last="Bae">Seung-Hee Bae</name><name sortKey="Bae, Seung Hee" sort="Bae, Seung Hee" uniqKey="Bae S" first="Seung-Hee" last="Bae">Seung-Hee Bae</name><name sortKey="Choi, Jong Youl" sort="Choi, Jong Youl" uniqKey="Choi J" first="Jong Youl" last="Choi">Jong Youl Choi</name><name sortKey="Choi, Jong Youl" sort="Choi, Jong Youl" uniqKey="Choi J" first="Jong Youl" last="Choi">Jong Youl Choi</name><name sortKey="Ekanayake, Jaliya" sort="Ekanayake, Jaliya" uniqKey="Ekanayake J" first="Jaliya" last="Ekanayake">Jaliya Ekanayake</name><name sortKey="Ekanayake, Jaliya" sort="Ekanayake, Jaliya" uniqKey="Ekanayake J" first="Jaliya" last="Ekanayake">Jaliya Ekanayake</name><name sortKey="Ekanayake, Saliya" sort="Ekanayake, Saliya" uniqKey="Ekanayake S" first="Saliya" last="Ekanayake">Saliya Ekanayake</name><name sortKey="Ekanayake, Saliya" sort="Ekanayake, Saliya" uniqKey="Ekanayake S" first="Saliya" last="Ekanayake">Saliya Ekanayake</name><name sortKey="Fox, Geoffrey" sort="Fox, Geoffrey" uniqKey="Fox G" first="Geoffrey" last="Fox">Geoffrey Fox</name><name sortKey="Fox, Geoffrey" sort="Fox, Geoffrey" uniqKey="Fox G" first="Geoffrey" last="Fox">Geoffrey Fox</name><name sortKey="Gunarathne, Thilina" sort="Gunarathne, Thilina" uniqKey="Gunarathne T" first="Thilina" last="Gunarathne">Thilina Gunarathne</name><name sortKey="Gunarathne, Thilina" sort="Gunarathne, Thilina" uniqKey="Gunarathne T" first="Thilina" last="Gunarathne">Thilina Gunarathne</name><name sortKey="Hughes, Adam" sort="Hughes, Adam" uniqKey="Hughes A" first="Adam" last="Hughes">Adam Hughes</name><name sortKey="Hughes, Adam" sort="Hughes, Adam" uniqKey="Hughes A" first="Adam" last="Hughes">Adam Hughes</name><name sortKey="Li, Hui" sort="Li, Hui" uniqKey="Li H" first="Hui" last="Li">Hui Li</name><name sortKey="Li, Hui" sort="Li, Hui" uniqKey="Li H" first="Hui" last="Li">Hui Li</name><name sortKey="Qiu, Judy" sort="Qiu, Judy" uniqKey="Qiu J" first="Judy" last="Qiu">Judy Qiu</name><name sortKey="Ruan, Yang" sort="Ruan, Yang" uniqKey="Ruan Y" first="Yang" last="Ruan">Yang Ruan</name><name sortKey="Ruan, Yang" sort="Ruan, Yang" uniqKey="Ruan Y" first="Yang" last="Ruan">Yang Ruan</name><name sortKey="Wu, Tak Lon" sort="Wu, Tak Lon" uniqKey="Wu T" first="Tak-Lon" last="Wu">Tak-Lon Wu</name><name sortKey="Wu, Tak Lon" sort="Wu, Tak Lon" uniqKey="Wu T" first="Tak-Lon" last="Wu">Tak-Lon Wu</name><name sortKey="Zhang, Bingjing" sort="Zhang, Bingjing" uniqKey="Zhang B" first="Bingjing" last="Zhang">Bingjing Zhang</name><name sortKey="Zhang, Bingjing" sort="Zhang, Bingjing" uniqKey="Zhang B" first="Bingjing" last="Zhang">Bingjing Zhang</name></country></tree></affiliations></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/CyberinfraV1/Data/Main/Exploration
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000892 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd -nk 000892 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Ticri/CIDE
|area= CyberinfraV1
|flux= Main
|étape= Exploration
|type= RBID
|clé= PMC:3040529
|texte= Hybrid cloud and cluster computing paradigms for life science applications
}}
Pour générer des pages wiki
HfdIndexSelect -h $EXPLOR_AREA/Data/Main/Exploration/RBID.i -Sk "pubmed:21210982" \
| HfdSelect -Kh $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd \
| NlmPubMed2Wicri -a CyberinfraV1
| This area was generated with Dilib version V0.6.25. |  |